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Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )^| Responsive to communication(s) filed on 22 December 2008 . 
2a )□ This action is FINAL. 2b)£3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^3 Claim(s) 1-43 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) |EI Claim(s) 1-43 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) L~H The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)^ accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 0 Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3.Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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Claim Status 

Claims 1-43 are pending 

Claim Rejections - 35 USC §102 

The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

Claims 1-43 are rejected under 35 U.S.C. 102(b) as being anticipated by Hongo (US 
4,712,248). 

Regarding claim 1, 17, 33, Hongo discloses: 

(a) at least one multimedia information input interface receiving said multimedia information; 
[Fig 1(a), iTV camera 1] 

(b) a computer processor [Fig 1(A), preliminary processing circuit 21], coupled to said at least 
one multimedia information input interface, 

receiving said multimedia information therefrom [col 3, lines 5-10, the output of video signal of 
the iTV camera 1 is converted into a binary signal by the preliminary processing circuit], 
processing said multimedia information by performing object extraction processing to create 
multimedia object descriptions from said multimedia information [col 1, lines 45-50, an object 
identifying apparatus for extracting feature data of an object to be identified and for comparing 
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the extracted feature data to feature data of reference objects to determine the identity of the 
object to be identified] 

and processing said created multimedia object descriptions by object hierarchy processing to 
generate create multimedia object hierarchy descriptions [Fig 9, col 9, lines 25-40] indicative of 
an organization of said object descriptions, 

wherein at least one description record including said multimedia object descriptions and said 
multimedia object hierarchy descriptions is created for content embedded within said multimedia 
information [col 3, lines 10-15, the feature extraction circuit 22 divides the binary-coded image 
into picture elements having a predetermined size, extracts data on each picture element or on a 
segment representing a series of picture elements in a horizontal scanning line]; and 
(c) a data storage system, operatively coupled to said processor, for storing said at least one 
description record [Fig 1(A), image memory 23, col 3, lines 1-30] 

Regarding claim 2, 18, 34, Hongo discloses wherein said multimedia information comprises 
image information, said multimedia object descriptions comprise image object descriptions, and 
said multimedia object hierarchy descriptions comprise image object hierarchy descriptions [Fig 
1(A), iTV camera 1] 

Regarding claim 3, 19, Hongo discloses: 

(a) image segmentation processing to segment each image in said image information into regions 
within said image; and 

(b) feature extraction processing to create one or more feature descriptions for one or more of 
said regions; whereby said created object descriptions comprise said one or more feature 
descriptions for one or more of said regions [col 3, lines 1-25] 
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Regarding claim 4, 20, 35, Hongo discloses wherein said one or more feature descriptions are 
selected from the group consisting of text annotations, color, texture, shape, size, and position 
[abstract] 

Regarding claim 5, 21, 36, Hongo discloses wherein said object hierarchy processing comprises 
physical object hierarchy organization to generate create physical object hierarchy descriptions 
of said image object descriptions that are based on spatial characteristics of said objects, such 
that said image object hierarchy descriptions comprise physical descriptions [col 1, lines 50-55] 
Regarding claim 6, 22, 37, Hongo discloses wherein said object hierarchy processing further 
comprises logical object hierarchy organization to create logical object hierarchy descriptions of 
said image object descriptions that are based on semantic characteristics of said objects, such that 
said image object hierarchy descriptions comprise both physical and logical descriptions [col 2, 
lines 5-10] 

Regarding claim 7, 23, Hongo discloses 

(a) image segmentation processing to segment each image in said image information into regions 
within said image; and 

(b) feature extraction processing to create object descriptions for one or more of said regions and 
wherein said physical hierarchy organization and said logical hierarchy organization create 
hierarchy descriptions of said object descriptions for said one or more of said regions [col 9, 
lines 25-40] 

Regarding claim 8, 24, 32, Hongo discloses an encoder receiving said image object hierarchy 
descriptions and said image object descriptions, and encoding said image object hierarchy 
descriptions and said image object descriptions into encoded description information, wherein 
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said data storage system is operative to store said encoded description information as said at least 
one description record [col 3, lines 5-25] 

Regarding claim 9, 25, 38, Hongo discloses video information, said multimedia object 
descriptions comprise video object descriptions including both event descriptions and object 
descriptions, and said multimedia hierarchy descriptions comprise video object hierarchy 
descriptions including both event hierarchy descriptions and object hierarchy descriptions [col 3, 
line 7] 

Regarding claim 10, 26, Hongo discloses 

(a) temporal video segmentation processing to temporally segment said video information into 
one or more video events or groups of video events and create event descriptions for said video 
events, 

(b) video object extraction processing to segment said one or more video events or groups of 
video events into one or more regions, and to create object descriptions for said regions; and 

(c) feature extraction processing to generate create one or more event feature descriptions for 
said one or more video events or groups of video events, and one or more object feature 
descriptions for said one or more regions; 

wherein said created video object descriptions include said event feature descriptions and said 
object descriptions [col 3, line 55] 

Regarding claim 1 1, 27, 39, Hongo wherein said one or more event feature descriptions are 
selected from the group consisting of text annotations, shot transition, camera motion, time and 
key frame, and wherein said one or more object feature descriptions are selected from the group 
consisting of color, texture, shape, size, position, motion, and time [col 3, lines 1-15] 
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Regarding claim 12, 13, 14, 15, 16, 28, 29, 30, 31, 40, 41, 42, 43, Hongo discloses wherein said 
object hierarchy processing comprises physical event hierarchy organization to generate create 
physical event hierarchy descriptions of said video object descriptions that are based on temporal 
characteristics of said video objects, such that said video hierarchy descriptions comprise 
temporal descriptions [col 1, lines 40-55] 

Response to Arguments 

Applicant's arguments filed 1/22/2008 have been considered but are moot in view of the 
new ground(s) of rejection. 

Contact Information 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Etienne P. LeRoux whose telephone number is (571) 272-4022. 
The examiner can normally be reached on Monday through Friday, 8:00 am - 4:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Apu Mofiz can be reached on (571) 272-4080. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Etienne P LeRoux/ 

Primary Examiner, Art Unit 2161 
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